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Abstract: Contrasted with other information carriers, such as speech and text, images contains larger amount of 
information, especially in sequential images, that is waiting to be exploited, in particular the dynamic information of 
correlation, difference, and temporal relationship between different frames. This dynamic information contributes a great 
deal in analysis of 4D images. This paper proposes a method for detecting dynamic information from sequential images, 
based on the rebuilding of their gray (position)-time function on direction lines, an approach that has been analyzed and 
studied extensively on the setting of various direction lines. This method is based on motion that is presented on 
sequential images. In particular, the method, Omni directional M-mode Echocardiography system, which we have studied 
extensively, will be described leading to a robust way of diagnosing heart diseases. 

Keywords: Rebuilding gray (position) ~ time function, tracking moving object on any directional line, echocardiography, 
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1. INTRODUCTION 

Sequential images, which composed of a stack of 2D 
static images in order arranged one by one, contain not only 
information that static images carry, but also information of 
motions hidden among frames, the latter appearing to be 
more important. To detect this information, optical flow 
detecting technique, which is a well-known method that 
applies computer vision theory to detection of dynamic 
information, analyzes the velocity field of each pixel in 
images from the viewpoint of statistics [1, 2]. According to 
our experience working on contour-based optical flow 
detecting [3], it always suffers from many difficulties on 
mathematic analysis when it is used to solve the problem on 
deformable objects and non-continuous view angles. 
Furthermore, there exist many statistic, conjectural and 
uncertain problems as far as specific understanding of 
velocity is concerned. Therefore many new detecting 
methods on Echocardiography Sequences [4-8] have been 
developed. In this study, we have searched for a new method 
to detect dynamic information from sequential images, 
which can rebuild gray (position) -time function (waveform) 
on direction lines to detect dynamic information on moving 
views, moving objects, and moving characteristic points in 
the images. 

Rebuilding a gray (position)-time function (waveform) 
on an arbitrary direction line is to reposition the recognized 
gray level of views, objects or characteristic points on that 
direction line and further to confirm its position by capturing 
the white dots on the line. Because these direction lines are 
artificial, the positions of those white dots can be confirmed. 
On the other hand, views and objects in the same frame of 
sequential images are relatively still, leading the white dots 



* Address correspondence to these authors at the Bio-Medial Engineering 
Institute ofFuzhou University, Fujian Fuzhou, 350002, China; 
Tel: 86-591-83727187; Fax: 86-591-87893589; 
E-mails: Chianglin@fzu.edu.cn, Liwei@fzu.edu.cn 



can be confirmed uniquely, while altering with the shifting 
frames of sequential images. Accordingly, with the certainty 
of the time interval between two neighboring frames, these 
white dots, which move originally on the direction lines, are 
extended with the time. Consequently the gray (position) ~ 
time waveform (function) on direction lines are obtained. 
Moving equation on direction lines are obtained when these 
dots are confirmed as moving objects. From the point of 
kinematics, the most important moving basic information 
data can be obtained. Therefore we can mine data in depth 
and find correlative new dynamic information with the 
combination of kinematics and dynamics. 

The movements of viewing angles, objects or 
characteristic points of practical images, however, are very 
complex. Therefore it is impossible that they simply move 
back and forth on a single direction line in most cases. As a 
result, the setting of direction lines of sequential images in a 
practical application should be a combination of many 
approaches so as to detect practical information and to 
engage the tracked target points with the combination of the 
settings of direction lines and certain algorithms. In this 
paper, two methods are developed to solve this problem. One 
is to catch the bright points in the ECG a form of single 
directional stepping translation with frames. Another is to 
obtain a non-functional motion model that is in turn applied 
on the direction line to capture functional moving points. 
Both methods are employed in practice [9, 10]. According to 
the moving situations of image objects, more direction lines 
can be set, leading to more data and new accurate dynamic 
information being mined and detected. 

In this paper, a method has been developed, which 
combines with certain algorithms, which focuses on the 
tracking of moving objects to set the direction lines, and is 
based on the movement of moving objects with the help of 
sequential images, composed of static images. 
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2. THE SETTING OF DIRECTION LINES TO 
REBUILD GRAY(POSITION)~TIME FUNCTION AND 
THE MINING OF THE DYNAMIC INFORMATION 

As discussed above, direction lines are used to track 
moving objects, so the optimal direction line is just the trace 
of a moving object being tracked. However, finding a 
direction line from images is difficult if not possible. To this 
end, what we can do is to set artificially some controllable 
and positionable direction lines which can capture the 
predefined view, objects and characteristic points. And then 
we can combine with some related algorithms to set 
direction lines to obtain a moving trace of the engaged 
objects. That is gray (position) -time function, which first- 
order differential is the velocity of movement, and second- 
order is its acceleration. Thus, we can mine various moving 
information. 

The method used to set direction lines can be classified 
into two kinds. The first is the setting of fixed direction lines 
and the second is the setting of direction line variables within 
frames. The details are given below. 

2.1. Setting of Fixed Direction Lines 

In this way, the position coordinates of direction lines to 
be set can be fixed and are invariable of frame numbers in 
the sequential images. The detailed setting can be divided 
into following three classes: 

• Single arbitrary direction line 

• Arbitrary number arbitrary direction lines 

• Multigroup arbitrary direction lines 

Multigroup arbitrary direction lines are a kind of 
arbitrary number arbitrary direction lines, which refers to 
that there are many groups of moving entities in a single 
image. With this setting, the settings of several direction 
lines are associated and constrained with each other, due to 
the fact that every moving entity in a group is coherent. For 
example, in a moving entity group of left ventricular short 
axis of cardio scans, the moving directions of every part tend 
to be centripetal. Thus, the direction of every direction line 
on every part of the group features commonly to be 
centripetal, which forms the association and restriction of 
each direction. 

2.2. Setting of Direction Line Variables with Frames 

With this setting, the position coordinates of a direction 
line are keeping altering and changing with the altering of 
sequential frames. It can be an arbitrary number on any 
arbitrary direction lines, as shown above. Therefore, this 
method is more complex than the setting of fixed direction 
lines. However, it may be a way to track more complex 
moving objects. As mensioned above, the major idea of the 
method is that all of the direction lines should be controllable 
and positionable, which requests the altering of this kind of 
direction lines should be regular and operated according to a 
certain algorithm so that they can be positioned in different 
frames of a set of sequential images. The detailed setting of 
altering direction lines can be divided into the following two 
subclasses: 

• Single directional stepping translation form with 
frames 
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• Complex moving form with frames 

The direction line of single directional stepping 
translation form keeps altering with frames. In this way, we 
can track the moving trace of an object, which moves very 
complexly within the range of R (the longest deviate distant 
to the center- line), as illustrated in Fig. (1). Where the 
length of the direction line is 2R, and the line is 
perpendicular to the centerline. The stepping translation of 
the direction line between two neighboring frames is d, and 
there is a direction line corresponding to each frame to track 
an object point. As a result, the moving trace of the object 
can be extracted from the gray (position) ~ time function 
(waveform), similar to sampling techniques. 
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Fig. (1). The setting of direction lines varied with frames in the 
tracking of moving objects. 

With this direction line in practice, the moving trace of 
the ECG object on the lower part of echocardiography can be 
obtained, illustrated in Fig. (2). We can join it synchronized 
to Omnidirectional M-mode Echocardiography (which will 
be introduced in the subsequent introduction) as their time 
reference. 




Fig. (2). The application of setting of direction lines variable with frames in 
the tracking of the bright spot object of ECG on echocardiography. 

2.3. Nonfunctional Motion and Functional Motion 

Normally, it is very complex to set a direction line of 
moving form within frames, but it may track more 
complicated moving traces, leading to a separation of 
number of important movements from the other movements, 
which is the main intension of this study. Using this method, 
we will evaluate Omnidirectional M-Mode Echocardio- 
graphy functional motion separation, the important issue 
when it comes to detect every structural function of cardiac 
to a great extent of precision. As the equipment that is used 
to non-traumatically detect cardiac moving image is given a 
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fixed position on an external body, the detected sequential 
motion images of one cardiac structure are only a general 
result, which consists of functional motion of this self 
structure (such as diastole and systole) and other 
nonfunctional motion (such as blood flow, breathe, the 
dragging between each structure.). Hence it can't single out 
the functional motion that can reflect the inherent 
information of one structure, the most obvious disadvantage 
of these medical dynamic images as illustrated in Fig. (3a). 
If the first and the last coordinate of the fixed coordinate 
line(x, y) is (105, 30; 136, 112), the cardiac structure of 
being captured from No.l, 14, 34 frame in the sequential 
images is in three different position, they are aortic front 
wall, ventricular septum and aortic front wall. On the other 



hand, under these three pictures, the corresponding 
rebuilding echocardiography is illustrated. Thanks to the 
influence of non-functional motion, the setting of fixed 
direction lines can't track point A of this structure nor 
capture the functional trace of this point at all times. In the 
current international standard, it is the momentous 
unfathomed problem to detect dynamic image information. 

By statistic moving of nonfunctional motion character 
points which are close to point A, we obtain the 
nonfunctional motion of these points in the sequential 
images of Fig. (3a), and act it on the direction line according 
to the principle of relative motion, consequently make the 
direction line move complicated. From Fig. (3b) we can 
seethe first and the last coordinate of direction line in 





Fig. (3a). Fixed direction lines capturing gray point and rebuilding M-mode cardiogram in sequential images (No.l frame, No.l4frame, 
No.34frame). 




Fig. (3b). Complex moving direction lines by the action of non-functional motion to capture gray point and rebuilding M-mode cardiogram 
in sequential images. 
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No.l, 14,34 frame is (105, 30; 136, 112) , (125, 28; 156, 
110), (93, 37; 124, 119). On the whole these points are all 
the positions of A of this structure, thus the rebuilding M- 
mode echocardiography is clearer, and synthesize the three 
lines near No. 14 frame into one line. 

3. THE SETTING MODEL OF FIXED ARBITRARY 
NUMBER DIRECTION LINES-OMNIDIRECTIONAL 
GRAY -TIME ECHOCARDIOGRAPHY WAVEFORM 

Omnidirectional gray ~ time echocardiography waveform, 
or called Omnidirectional M-mode Echocardiography, is the 
most typical example of the setting of fixed arbitrary number 
direction line, and is a fruitful method in our research. 
According to the rebuilding principle of Omnidirectional 
gray ~ time waveform, if the direction of a sample line 
selected is the same as the actual motion direction, the gray 
intensity data of the pixels on them (e.g. B) can generate 
gray ~ time waveform combined with corresponding 
temporal relationships. As shown in Fig. (4), the gray ~ time 
waveform of three lines is rebuilt from long and short axis 
echocardiography respectively and represents motion 
waveform of the wall of ventricle structure, which is because 
these gray values can stand for the boundary of cardiac 
structure in ultrasound images. Furthermore, all these 




resulted M-mode echocardiography are synchronised in 
time, which reflects the movements of different parts in 
different cardiac structures. 

As for Omnidirectional gray ~ time waveform detected 
from echocardiography sequential images, we can select 
arbitrary direction of any parts in any structures to set sample 
line (direction line), which can be used by to follow the 
motion tracks of this part. Thus, sample lines are not 
confined by the direction of ultrasound wave- line any longer. 
Consequently a substantial improvement in detection 
accuracy is obtained. Moreover, we can compare the motion 
parameter of every echocardiography and their temporal- 
phase relationships, because any arbitrary number sample 
line whose waveform is synchronous in time can be selected. 
As a result, the idea of Omnidirectional M-mode 
Echocardiography Waveform System has been patented by 
the National Invent Patent of China (No. ZL 98 125713.5). 

3.1. LEJ-1 Omnidirectional M-Mode Echocardiography 
System 

Subsequently, we manufactured LEJ-1 Omnidirectional 
M-mode echocardiography system with the setting of fixed 
direction lines. The system facilitates two functions, namely 




Fig. (4). The setting of three fixed direction lines rebuilding gray ~ time echocardiography wave- form of motion orientation in long axis (the 
right) and short axis (the left) of cardiac structure. 




Fig. (5). Omnidirectional gray -time echocardiography waveform in automatic operation. 
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Fig. (6). A lot of Omnidirectional of gray ~ time echocardiography Waveform in manual operation Comparative waveform with ECG. 



automatic operation and manual operation. In the former, 
when confirmed a center, 12 oriented direction lines will 
give out every 30 angles, which looks like a clock panel. 
Then their gray ~ time waveform of specified duration will 
expend in screen or be printed on laser printer paper, as 
given in Fig. (5). This kind of operation is mainly suited for 
left ventricle short axis sequence images whose motion is 
mainly acentric and centripetal. 

The second operation is manual operation. In accordance 
with any positions, any structures in echocardiography 
images, based on their motion direction, operator can select 
any direction sample line with mouse and set manually to 
build their Omnidirectional echocardiography gray ~ time 
waveform lasting for 5 seconds. In this operation, there will 
be a preview (echocardiography waveform) displayed on the 
monitor to give a choice on whether a sample line is 
selected. 

Regardless which operation is carried out, a series of 
resulted M-mode echocardiography waveforms 
corresponding to a sampling line can be chosen to perform 
further detection for various parameters. Parameters such as 
amplitude, velocity, compressing peak velocity and the 
quality of thickness, which related to the motion of cardiac 
structure, are detected from appointed sample line in 
echocardiography waveform as demonstrated in Fig. (6). 
According to the description above, we rebuild ECG below 
echocardiography which is synchronized with all of 
Omnidirectional M-mode echocardiography waveform so as 
to compare Omnidirectional gray ~ time waveform with 
ECG (see Fig. 2). 



3.2. Clinical Application 

Having passed the physical test conducted by Fujian 
Province Central Test Institute in China and by several 
hospitals when being clinically practiced, the method 
detailed above is verified that resulting detection data are 
accurate, scientific, and trustworthy. 

Consequently, we have developed the LE J- 1 Omnidirect- 
ional M-mode echocardiography system and applied it to 
clinical practice in the Fujian Provincial Research Institute 
for Cardiovascular Diseases [11-14]. Some of clinical 
practice reports detected by Fujian Provincial Research 
Institute for Cardiovascular Diseases are given in Table 1. 

In Table 1, the statistical values of t counter- sample, 
which are measured on Omnidirectional M-mode 
echocardiography system, are both smaller than the statistical 
values t with a degree of freedom of 29 and a probability of 
0.05. On the point of statistics, the result is reasonable. 
Otherwise, between Omnidirectional M-mode echocardio- 
graphy and original B-scan echocardiography, according to 
the comparative experimental standards practiced in clinic, 
the data at the same location are very similar. In fact, their 
difference is statistically insignificant. 

The system has been applied into clinical practice in 
Chinese People's Liberation Army (CPLA) General Hospital 
(301 Hospital, Beijing) for 3 years [15-18]. They summarize 
the data and the experiments in their own clinical practice 
according to the function of the system. The related research 
is mainly on the assessment of left ventricle regional systolic 
and diastolic functions, atria septal motion and atria load 
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Table 1. A Number of Clinical Practice Report Detected by Fujian Provincial Research Institute for Cardiovascular Diseases 



Thickness of posterior wall at the end of diastole of left ventricle on long axis 


Cases: ^ = 30 


(v = n - 1 = 29); t = 1 .7 1 3 8 < f a05(29) = 2 .045 


Diastolic velocity of posterior wall on long axis 


Cases: n = 30 


(y=n - 1 = 29); t= 0.2372 < f a05(29) = 2.045 


Inner diameter of the aorta at the end of diastole on the long axis 


Cases: ^ = 30 


(v = n - 1 = 29); t = 1 .45 6 1< t 0 05(29) = 2 .045 


Systolic velocity of the aorta on the long axis 


Cases: /? = 30 


(v= n - 1 = 29); t= 1 .5205 < f a05(29) = 2.045 


Thickness of posterior wall at the end of the diastole on the short axis 


Cases: ^ = 30 


0 = «-l= 29); t = 1 .8963 < f a05(29) = 2.045 


Thickness of posterior wall at the end of the systole on the short axis 


Cases: ^ = 30 


(v=n - 1 = 29); t = 1 .3794 < t 005{29) = 2.045 
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Acceleration c^t) = d 2 S/ dt (cm/s 2 )— 




Fig. (7). Omnidirectional M-mode echocardiography images with its velocity and acceleration functions. 



relation, observation of pulmonary artery motion curve of 
pulmonary hypertension, etc. 

The ultrasound department of the General Hospital of 
CPLA has made a detailed research on regional motion 
function of heart with the system. Hundreds of cases with 
varying age, normal and abnormal cardiovascular functions 
are all diagnosed. The application of the Omnidirectional M- 
mode echocardiography in assessment of left ventricle 
regional diastolic function may be recommended as a 
diagnostic method of examining heart degradation earlier 
when the global functionality is correct while local 
functionality is lightly degrading. 

4. THE DATA MINING OF DYNAMIC INFORMAT- 
ION IN SEQUENTIAL IMAGES 

The most extinctive difference between our Omnidi- 
rectional gray ~ time waveform system and Anatomy M- 
mode Echocardiography [19] is that we have obtained 



synchronous Omnidirectional gray ~ time waveform group 
on any parts, any direction and arbitrary columns (corres- 
ponding to arbitrary sample lines) based on dynamic infor- 
mation detected from sequential images with synchronous 
ECG, whereas. The anatomical M-mode echocardiography 
shows only two or three images simultaneously even when 
they come from a different cardiac period. 

Compare with Tissue Doppler echocardiography [20], 
our system has some characteristics as specified in the 
follows [21]: 

• the accelerator comes from differential coefficient to 
athletic function directly not from frequency 
excursion indirectly. 

• the orientation can changes freely. 

• Groups of arbitrary direction lines in any part of any 
structure can be set and presented corresponding to 
Omnidirectional echocardiography images with 
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synchronous ECG so as to be compatible with 
waveforms and temporal phases. 

The dynamic analysis here we made in sequential images 
is similar to amplifier analysis made for several object units 
in static images. One example is that an image of streets in a 
static whole scene may only contain lines. However, when 
further amplified, these lines can display cars, shops, or other 
details in these streets. Under the similar pattern, the 
sequential images show the "whole scene" of every cardiac 
structure from its full view. It can be implemented by the 
way of Omnidirectional gray ~ time wave- form detecting 
method to acquire detailed knowledge of the motion status, 
namely dynamic details of any parts in any structures 
(including motion waveform, amplitude, velocity, and 
accelerate, etc. when the point's movement being along with 
the time). The wave- form, which we obtained, is the motion 
trace or motion equation of the detecting part in cardiac 
structure along with time. On the base of it, one-order timing 
of differential of Omnidirectional gray ~ time wave- form is 
ds 

the velocity ( — = v ) of detecting movement of cardiac 
dt 

d 2 s 

structure and two-order is acceleration ( — - = a ). As 

dt 

follows Fig. (7). 

In summary, it is the most important to obtain new 
dynamic information in the hemodynamics study. So we 
implement the data mining of new dynamic information 
from every cardiac structure from sequential images. We 
have applied the method to the analysis of cardiac B- 
scan/color scan (or echocardiography) sequential images. In 
fact, this motion analysis method on sequential images can 
also be employed to many other medical sequential images 
such as X-ray, coronary-contrast, etc. Consequently, this 
method for motion analysis can also be improved and 
developed further to fit the characteristics of each imaging 
modality. 

5. CONCLUSION AND DISCUSSION 

With the advanced development of science and 
technology, the method detailed in the paper and used to 
analyze dynamic information is becoming much profounder. 
The main methods currently used can be classified as a 
viewpoint-based approach, which is employed on sequential 
images, focusing on dynamic objects and on movement. 
While based on sequential images, the method proposed in 
this paper constitutes all its syntaxes, such as the setting of 
the direction lines and the algorithm combined, for moving 
objects principally. So it has some characteristics such as 
understanding motion details of each cardiac structure, 
confirming moving information data, and correcting 
detecting results. To this end, the prior knowledge bases are 
required in order to set direction lines easier to analyze the 
motion information. 

At present, it is required that all pixels are relatively 
stationary in a single sequential image. The time interval 
between two neighboring frames is 1/25 second in the PAL 
system and 1/30 second in the NTSC system, which is a very 



long time for usual objects. So, if the time of frame interval 
is reduced farther, the dynamic information detection of 
higher-speed movement can be obtained with this proposed 
method. 

In addition, the direction lines used above are all straight 
lines. In the future, if they are modified into controllable, 
positionable, and non-linear direction lines, it will be easier 
to track more complex movement leading to the improve- 
ment of this approach further. 
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